System for analyzing workflow and detecting inactive operators and methods of using the same

ABSTRACT

Provided is an improved apparatus and method for identifying inefficient workers in a workplace. The apparatus receives voice commands from a worker in the workplace and transforms the voice commands into text commands by converting audio signals associated with the voice commands to one or more points in data that are stored in a database. The apparatus further creates a dialogue-stream including the text commands and a time of receipt of each voice command associated with the respective text command. Further, a workflow record may be created from the dialogue-stream and may be used to generate an efficiency metric for the worker. Both the dialogue-stream and workflow record can be used to monitor the efficiency of the worker and implement effective changes to the workflow process.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. application Ser. No.15/786,971, filed Oct. 18, 2017, the contents of which are herebyincorporated herein in their entirety by reference.

BACKGROUND

Management has little visibility into the actual activity on the floorof a company where work is being done. Whether in a warehouse,manufacturing facility, retail store, etc., supervisors and othermanagement personnel have a difficult time monitoring daily activitiesof the respective business, particularly when dealing with largerfacilities with numerous employees. However, it is in these dailyactivities where costs can rise and efficiencies can decrease.

Applicant has identified a number of deficiencies and problemsassociated with conventional systems and associated devices and methodsfor capturing workflow performance. Through applied effort, ingenuity,and innovation, many of these identified problems have been solved bydeveloping solutions that are included in embodiments of the presentinvention, many examples of which are described in detail herein.

SUMMARY

The present disclosure relates to system and method for identifyinginefficient workers in a workplace. In a first exemplary embodiment, avoice controlled apparatus to identify inefficient workers in aworkplace is provided. The voice controlled apparatus may include aprocessor. The processor may receive voice commands from a worker in theworkplace. The processor may further transform the voice commands intotext commands by converting audio signals associated with the voicecommands to one or more points in data that are stored in a database.The processor may create a dialogue-stream including the text commandsand a time of receipt of each voice command associated with therespective text command. The processor may cause the dialogue-stream tobe transmitted to a server. Further, a workflow record may be createdfrom the dialogue-stream and may be used to generate an efficiencymetric. The workflow record may include individual tasks and may becreated by identifying a task associated with any one of the voicecommands. The task may indicate that the worker has at least started toperform the task and the task is related to an efficiency of the worker.

In some embodiments, a dialogue-stream including the text commands and atime of receipt of each voice command associated with the respectivetext command may be gathered into a single collection organized by thetime of receipt. Such gathering and organizing may be performed by theprocessor and/or another device, system, or component. For instance, aback-end server or cloud based system may receive the dialogue-streamand organize and collect the text commands by time of receipt (of theassociated voice command at the user device).

In some embodiments, the processor may receive button commands manuallyinputted by the worker in the workplace. The processor may incorporatethe button commands into the text commands by identifying one or morepoints in data stored in the database or another database related toeach of the button commands. The processor may further incorporate thetext commands from the button commands into the dialogue-stream and/oranother device, system, or component may do so. For instance, a back-endserver or cloud based system may receive the text command associatedwith the button command and organize the text command by time of receipt(of the associated button command at the user device) into thedialogue-stream.

In some embodiments, the processor may provide speech to the worker byconverting one or more points in data to one or more audio signals to beconverted to speech via a speaker.

In some embodiments, the voice controlled apparatus may further includea microphone and a speaker.

In some embodiments, the processor may further cause the dialogue-streamto be transmitted to a supervisor device. The supervisor device may becontrolled by a supervisor of the worker.

In some embodiments, the dialogue-stream may be used to prepare one ormore alarms associated with a maximum duration of a particular task ofthe workflow record.

In some embodiments, the dialogue-stream may be further used todetermine the rate of interaction of the worker with the apparatus.

In further embodiments, the dialogue-stream may be further used toidentify periods of non-activity of the worker with the apparatus.

The present disclosure further relates to a method of identifyinginefficient workers in a workplace. The method may include receiving adialogue-stream from a user device. The dialogue-stream may include textcommands and a time associated with each text command. Further, the textcommands may be formed by transforming voice commands from a particularuser into the text commands by converting audio signals associated withthe voice commands to one or more points in data that are stored in adatabase. The method may further include creating a workflow record forthe particular user by identifying a task associated with any one of thevoice commands from the particular user based on each text commandassociated with the respective voice command and determining a durationof the task. Further, the workflow record may be used to generate anefficiency metric.

The method may include gathering the text commands into a singlecollection organized by time of receipt of the associated voice commandand/or button command.

In some embodiments, the method may further include recognizing tags inthe dialogue-stream. The tags may indicate that an associated textcommand in the dialogue-stream originated from a user as opposed to theuser device. Further, the method may include removing items in thedialogue-stream that are not associated with the tags to create auser-initiated dialogue-stream. In some embodiments, the method mayfurther include determining a rate of interaction of the particular userby calculating a number of text commands over a period of time in theuser-initiated dialogue-stream.

In some embodiments, identifying a task associated with any one of thevoice commands from the particular user based on each text commandassociated with the respective voice command may include determining acurrent state of the user device, determining a next state of the userdevice, identifying a trigger to move to the next state of the userdevice, and adding the task to the workflow record after identifying thetrigger. In some embodiments, the current state may be determined by theearliest received voice command and associated text command or a voicecommand received from the user device that has not been associated withat least one task. The next state may be determined by identifying atask intended to follow the current state from the database or anotherdatabase. Further, the trigger may include a dialogue of interestrelated to the next state.

In some embodiments, determining the duration of the at least one taskmay include identifying a text command or a voice command received fromthe user device in the dialogue-stream associated with a start of thetask in the database or another database, identifying a text command ora voice command received from the user device in the dialogue-streamassociated with an end of the task in the database or another database,and determining a time between the start of the task and the end of thetask.

In some embodiments, the method may further include determining anaverage duration or central tendency of the duration of tasks associatedwith the voice commands by averaging the duration of each task over astatistically significant number of repetitions of the task.

In some embodiments, the method may further include creating an alarmassociated with the task. The alarm may notify a supervisor device thatthe particular user has exceeded a maximum duration of the task bycausing an indication to be transmitted to the supervisor device whenthe duration of the task exceeds the maximum duration of the task in theworkflow record.

In some embodiments, the method may further include transmitting theworkflow record for the particular user to be transmitted to asupervisor device, wherein the supervisor device is controlled by asupervisor of the user.

In some embodiments, the method may further include receivingdialogue-streams from user devices. Each dialogue-stream may includetext commands and a time associated with each text command. The textcommands may be formed by transforming voice commands into the textcommands by converting audio signals associated with the voice commandsto one or more points in data that are stored in the database or anotherdatabase. The method may further include creating a workflow records forthe users by identifying a task for a voice command in each workflowrecord and determining a duration of each task of each workflow record,and comparing the duration of each task of each workflow record.

In some embodiments, the method may further include filtering theworkflow records for a single task, and determining an average durationof the single task over the workflow records, a minimum duration of thesingle task over the workflow records, or a maximum duration of thesingle task over the workflow records. In some embodiments, the methodmay further include creating an alarm associated with the single task.The alarm may notify a supervisor device that a user has exceeded a setduration of the single task by transmitting an indication to thesupervisor device when the duration of the single task exceeds the setduration in the workflow record of the user. The alarm may notify asupervisor device that a user's average duration for the task hasexceeded a set duration of the task by transmitting an indication to thesupervisor device when the average duration of the task exceeds the setduration in the workflow record of the user. In some embodiments, thealarm may be transmitted to the user device.

In some embodiments, the method may further include identifying a periodof non-activity of a particular user by identifying a period of timewhere the rate of interaction decreases to a defined value and causingan indication to be transmitted to a supervisor device notifying asupervisor of the particular user of the occurrence of the period ofnon-activity.

The present disclosure further relates to a method of identifyinginefficient workers in a workplace. The method may include receiving aworkflow record from a user device or a server. The workflow record mayinclude text commands and a time associated with each text command. Thetext commands may be formed by transforming voice commands into the textcommands by converting audio signals associated with the voice commandsto one or more points in data that are stored in a database. Further,the workflow record may be used to generate an efficiency metric. Themethod may further include acknowledging receipt of the workflow recordand creating an alarm associated with a task performed in the workflowrecord. The alarm may notify a supervisor device that a user hasexceeded a set duration of the task and/or that the user's averageduration for the task has exceeded a set duration or limit on theduration of the task. The method may include transmitting an indicationto the supervisor device when the duration of the task exceeds the setduration in the workflow record, or initiating an action to improve theworkflow record and incorporating a note to the workflow recordrecording the initiated action.

In some embodiments, the workflow record may include tasks associatedwith the text commands and each task may be associated with a durationof the respective task.

In some embodiments, the acknowledging receipt of the workflow recordmay include transmitting an indication to the user device or server fromwhich the workflow record was received, the indication indicating thatthe workflow record was received.

In some embodiments, the set duration may be derived from or related toan average duration of the task as performed by users.

In some embodiments, the action to improve the workflow record mayinclude notifying a particular user that the particular user isinefficient by causing an indication to be transmitted to the particularuser device operated by the particular user, the indication indicatingthat the particular user is inefficient.

In some embodiments, the action to improve the workflow record mayinclude requesting additional materials, workers, or a combinationthereof to perform the task by transmitting an indication to a server,the indication indicating that additional materials, workers, or acombination thereof are needed to perform the task.

The above summary is provided merely for purposes of summarizing someexample embodiments illustrating some aspects of the present invention.Accordingly, it will be appreciated that the above-described embodimentsare merely examples and should not be construed to narrow the scope orspirit of the invention in any way. It will be appreciated that thescope of the invention encompasses many potential embodiments inaddition to those herein summarized, some of which will be furtherdescribed below.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings illustrate embodiments of the invention, and,together with the specification, including the general description aboveand the detailed description which follows, serve to explain thefeatures of the present invention.

FIG. 1 illustrates a schematic of a workflow performance systemaccording to an example embodiment;

FIG. 2 illustrates a schematic block diagram of a workflow performancesystem according to an example embodiment;

FIG. 3 illustrates a schematic block diagram of a workflow databaseaccording to an example embodiment;

FIG. 4 illustrates a schematic block diagram of a workflow systemaccording to an example embodiment;

FIG. 5 illustrates an exemplary user device according to an exampleembodiment;

FIG. 6 illustrates a block diagram of an exemplary user device accordingto an example embodiment;

FIG. 7 illustrates an exemplary user device according to an exampleembodiment;

FIG. 8 is a flowchart illustrating a method for providing speechdialogue between a user and a device, according to an exampleembodiment;

FIG. 9 is a flowchart illustrating a method for identifying inefficientworkers in a workplace, according to an example embodiment; and

FIG. 10 is a flowchart illustrating a method for identifying inefficientworkers in a workplace, according to an alternate example embodiment.

DETAILED DESCRIPTION

The present invention now will be described more fully hereinafter withreference to the accompanying drawings in which some but not allembodiments of the inventions are shown. Indeed, these inventions may beembodied in many different forms and should not be construed as limitedto the embodiments set forth herein; rather, these embodiments areprovided so that this disclosure will satisfy applicable legalrequirements. Like numbers refer to like elements throughout. As usedherein, terms such as “front,” “rear,” “top,” “outside,” “inside,” etc.are used for explanatory purposes in the examples provided below todescribe the relative position of certain devices or portions ofdevices. Terminology used in this patent is not meant to be limitinginsofar as devices described herein, or portions thereof, may beattached or utilized in other orientations.

The term “comprising” means including but not limited to, and should beinterpreted in the manner it is typically used in the patent context.Use of broader terms such as comprises, includes, and having should beunderstood to provide support for narrower terms such as consisting of,consisting essentially of, and comprised substantially of.

The phrases “in one embodiment,” “according to one embodiment,” and thelike generally mean that the particular feature, structure, orcharacteristic following the phrase may be included in at least oneembodiment of the present invention, and may be included in more thanone embodiment of the present invention (importantly, such phrases donot necessarily refer to the same embodiment).

The word “exemplary” is used herein to mean “serving as an example,instance, or illustration.” Any implementation described herein as“exemplary” is not necessarily to be construed as preferred oradvantageous over other implementations.

The terms “about” or “approximately” or the like, when used with anumber, may mean that specific number, or alternatively, a range inproximity to the specific number, as understood by persons of skill inthe art field.

If the specification states a component or feature “may,” “can,”“could,” “should,” “would,” “preferably,” “possibly,” “typically,”“optionally,” “for example,” “often,” or “might” (or other suchlanguage) be included or have a characteristic, that particularcomponent or feature is not required to be included or to have thecharacteristic. Such component or feature may be optionally included insome embodiments, or it may be excluded.

As used herein, the term “transmitter” refers to any component that cangenerate radio waves for communication purposes while a “receiver” isused to generally refer to any component that can receive radio wavesand convert the information into useable form. A “transceiver” generallyrefers to a component that can both generate radio waves and receiveradio waves and is thus contemplated when either a transmitter or areceiver is discussed.

The term “processor” is used herein to refer to any programmablemicroprocessor, microcomputer or multiple processor chip or chips thatcan be configured by software instructions (applications) to perform avariety of functions, including the functions of the various embodimentsdescribed above. In some devices, multiple processors may be provided,such as one processor dedicated to wireless communication functions andone processor dedicated to running other applications. Softwareapplications may be stored in the internal memory before they areaccessed and loaded into the processors. The processors may includeinternal memory sufficient to store the application softwareinstructions. In many devices the internal memory may be a volatile ornonvolatile memory, such as flash memory, or a mixture of both. Thememory can also be located internal to another computing resource (e.g.,enabling computer readable instructions to be downloaded over theInternet or another wired or wireless connection).

For the purposes of this description, a general reference to “memory”refers to memory accessible by the processors including internal memoryor removable memory plugged into the device and memory within theprocessors themselves. For instance, memory may be any non-transitorycomputer readable medium having computer readable instructions (e.g.,computer program instructions) stored thereof that are executable by aprocessor.

The term “computing device” is used herein to refer to any one or all ofprogrammable logic controllers (PLCs), programmable automationcontrollers (PACs), industrial computers, desktop computers, personaldata assistants (PDAs), laptop computers, tablet computers, smart books,palm-top computers, personal computers, smartphone, headset, smartwatch,and similar electronic devices equipped with at least a processorconfigured to perform the various operations described herein. Devicessuch as smartphones, tablets, headsets, and smartwatches are generallycollectively referred to as mobile devices.

The term “server” is used to refer to any computing device capable offunctioning as a server, such as a master exchange server, web server,mail server, document server, or any other type of server. A server maybe a dedicated computing device or a computing device including a servermodule (e.g., running an application which may cause the computingdevice to operate as a server). A server module (e.g., serverapplication) may be a full function server module, or a light orsecondary server module (e.g., light or secondary server application)that is configured to provide synchronization services among the dynamicdatabases on computing devices. A light server or secondary server maybe a slimmed-down version of server type functionality that can beimplemented on a computing device, such as a smart phone, therebyenabling it to function as an Internet server (e.g., an enterprisee-mail server) only to the extent necessary to provide the functionalitydescribed herein.

In a workplace environment, an employee or a worker is often assigned aset of tasks and/or a job to work upon which is overseen by a supervisoror a manager to monitor the worker's performance and productivity.However, since a supervisor may have a high number of workers to monitorand/or may have other responsibilities or duties to tend to, there isgenerally a lack of visibility or ability to supervise all workerseffectively. In light of these problems in oversight by supervisors,workers may be tempted to remain inactive or idle, and/or may workslower than desired by management. Thus, there is a need for betterworksite monitoring, and specifically, to track periods of inactivity bya worker.

The present disclosure relates to a method for identifying inefficientworkers in a workplace. In a task based workflow solution, a workerand/or a user may interact with a user device throughout, periodically,or at specific intervals while performing the task. For example, a usermay receive specific instructions or voice commands from a user device,such as a headset, for executing a task. In response, the user mayprovide inputs, such as speech inputs, to the user device as requested.Thus, a user may be required to provide inputs to the user deviceperiodically for executing the workflow task efficiently. Hence, if auser does not interact with the user device as expected, or within aspecific duration, it may be determined that the user might be inactiveand/or working slower than desired. The present disclosure providessystems and methods for monitoring the interaction between the user andthe user device to identify inefficient workers. Specifically, adialogue-stream of worker activities may be created based on the voicecommands received from each worker and a time of receipt of each voicecommand. The dialogue-stream may be analyzed and transformed into aworkflow record. The workflow record may be used to determine anefficiency of the worker related to a given task and/or sub-task.Specifically, the dialogue-stream and workflow record may provideinformation regarding a duration of time taken by the worker to completethe task and/or the sub-task, rate of activity of the worker, and thelike. Further, alarms and/or alerts may be provided to a supervisorand/or other management positions when an efficiency parameter fallsoutside of the parameters or thresholds set to define an efficientworker. The supervisor may then take an appropriate action to maintainemployee productivity to desired levels. Thus, the present disclosureprovides efficient workplace monitoring methods and systems.

The components illustrated in the figures represent components that mayor may not be present in various embodiments of the invention describedherein such that embodiments may include fewer or more components thanthose shown in the figures while not departing from the scope of theinvention.

While the present disclosure focuses on workflow performance systemsgenerally in an employment setting, one or more embodiments of thepresent disclosure may be used for other purposes or in otherenvironments, such as to connect people of other organizations,families, schools, etc. and provide visibility of performance oractivity to those involved. The present disclosure may be particularlybeneficial when it is desired to connect remote users or provide accessto information to remote users that are not able to physically witness aperson's performance or activity.

FIG. 1 shows system 100 including an example network architecture for asystem, which may include one or more devices and sub-systems that areconfigured to implement some embodiments discussed herein. For example,system 100 may include server 160, which can include, for example, thecircuitry disclosed in FIGS. 2-4 , a server, or database, among otherthings (not shown). The server 160 may include any suitable networkserver and/or other type of processing device. In some embodiments, theserver 160 may receive requests and transmit information or indicationsregarding such requests to operator devices 110A-110N and/or one or moresupervisor devices 150.

Server 160 can communicate with one or more operator devices 110A-110Nand/or one or more supervisor devices 150 via network 120. In thisregard, network 120 may include any wired or wireless communicationnetwork including, for example, a wired or wireless local area network(LAN), personal area network (PAN), metropolitan area network (MAN),wide area network (WAN), or the like, as well as any hardware, softwareand/or firmware required to implement it (such as, e.g., networkrouters, etc.). For example, network 120 may include a cellulartelephone, an 802.11, 802.16, 802.20, and/or WiMax network. In someembodiments, Bluetooth may be used to communicate between devices.Further, the network 120 may include a public network, such as theInternet, a private network, such as an intranet, or combinationsthereof, and may utilize a variety of networking protocols now availableor later developed including, but not limited to TCP/IP based networkingprotocols.

Operator devices 110A-110N, supervisor device 150, and/or server 160 mayeach be implemented as a computing device, such as a personal computerand/or other networked device, such as a cellular phone, tabletcomputer, mobile device, point of sale terminal, inventory managementterminal etc., that may be used for any suitable purpose in addition topresenting the interface to facilitate buying items and/or offeringitems for sale. The depiction in FIG. 1 of “N” members is merely forillustration purposes. Further, while only one supervisor device 150 isillustrated in FIG. 1 , in some embodiments, multiple or a plurality ofsupervisor devices 150 may be connected in the system. Any number ofusers, operators, or supervisors may be included in the system 100. Inone embodiment, the operator devices 110A-110N and/or supervisor devices150 may be configured to display an interface on a display of therespective device for viewing, creating, editing, and/or otherwiseinteracting with the server. According to some embodiments, the server160 may be configured to display the interface on a display of theserver 160 for viewing, creating, editing, and/or otherwise interactingwith information on the server 160. In some embodiments, an interface ofoperator devices 110A-110N and/or supervisor device 150 may be differentfrom an interface of a server 160. Various components of the presentsystem may be performed on one or more of the operator devices110A-110N, supervisor device 150, or server 160. System 100 may alsoinclude additional client devices and/or servers, among other things.

FIG. 2 shows a schematic block diagram of circuitry 200, some or all ofwhich may be included in, for example, operator devices 110A-110N,supervisor device 150, and/or server 160. Any of the aforementionedsystems or devices may include the circuitry 200 and may be configuredto, either independently or jointly with other devices in a network 120perform the functions of the circuitry 200 described herein. Asillustrated in FIG. 2 , in accordance with some example embodiments,circuitry 200 can includes various means, such as processor 210, memory220, communications module 240, and/or input/output module 250. In someembodiments, workflow database 300 and/or workflow system 400 may alsoor instead be included. As referred to herein, “module” includeshardware, software and/or firmware configured to perform one or moreparticular functions. In this regard, the means of circuitry 200 asdescribed herein may be embodied as, for example, circuitry, hardwareelements (e.g., a suitably programmed processor, combinational logiccircuit, and/or the like), a computer program product comprisingcomputer-readable program instructions stored on a non-transitorycomputer-readable medium (e.g., memory 220) that is executable by asuitably configured processing device (e.g., processor 210), or somecombination thereof.

Processor 210 may, for example, be embodied as various means includingone or more microprocessors with accompanying digital signalprocessor(s), one or more processor(s) without an accompanying digitalsignal processor, one or more coprocessors, one or more multi-coreprocessors, one or more controllers, processing circuitry, one or morecomputers, various other processing elements including integratedcircuits such as, for example, an ASIC (application specific integratedcircuit) or FPGA (field programmable gate array), or some combinationthereof. Accordingly, although illustrated in FIG. 2 as a singleprocessor, in some embodiments processor 210 comprises a plurality ofprocessors. The plurality of processors may be embodied on a singlecomputing device or may be distributed across a plurality of computingdevices collectively configured to function as circuitry 200. Theplurality of processors may be in operative communication with eachother and may be collectively configured to perform one or morefunctionalities of circuitry 200 as described herein. In an exampleembodiment, processor 210 is configured to execute instructions storedin memory 220 or otherwise accessible to processor 210. Theseinstructions, when executed by processor 210, may cause circuitry 200 toperform one or more of the functionalities of circuitry 200 as describedherein.

Whether configured by hardware, firmware/software methods, or by acombination thereof, processor 210 may comprise an entity capable ofperforming operations according to embodiments of the present inventionwhile configured accordingly. Thus, for example, when processor 210 isembodied as an ASIC, FPGA or the like, processor 210 may comprisespecifically configured hardware for conducting one or more operationsdescribed herein. Alternatively, as another example, when processor 210is embodied as an executor of instructions, such as may be stored inmemory 220, the instructions may specifically configure processor 210 toperform one or more algorithms and operations described herein, such asthose discussed in connection with FIGS. 1-10 .

Memory 220 may comprise, for example, volatile memory, non-volatilememory, or some combination thereof. Although illustrated in FIG. 2 as asingle memory, memory 220 may comprise a plurality of memory components.The plurality of memory components may be embodied on a single computingdevice or distributed across a plurality of computing devices. Invarious embodiments, memory 220 may comprise, for example, a hard disk,random access memory, cache memory, read only memory (ROM), erasableprogrammable read-only memory (EPROM) & electrically erasableprogrammable read-only memory (EEPROM), flash memory, magneticcassettes, magnetic tape, magnetic disk storage or other magneticstorage devices, a compact disc read only memory (CD-ROM), digitalversatile disc read only memory (DVD-ROM), an optical disc, circuitryconfigured to store information, or some combination thereof. Memory 220may be configured to store information, data (including item data and/orprofile data), applications, instructions, or the like for enablingcircuitry 200 to carry out various functions in accordance with exampleembodiments of the present invention. For example, in at least someembodiments, memory 220 is configured to buffer input data forprocessing by processor 210. Additionally or alternatively, in at leastsome embodiments, memory 220 is configured to store program instructionsfor execution by processor 210. Memory 220 may store information in theform of static and/or dynamic information. This stored information maybe stored and/or used by circuitry 200 during the course of performingits functionalities.

Communications module 240 may be embodied as any device or meansembodied in circuitry, hardware, a computer program product comprisingcomputer readable program instructions stored on a computer readablemedium (e.g., memory 220) and executed by a processing device (e.g.,processor 210), or a combination thereof that is configured to receiveand/or transmit data from/to another device and/or network, such as, forexample, a second circuitry 200 and/or the like. In some embodiments,communications module 240 (like other components discussed herein) canbe at least partially embodied as or otherwise controlled by processor210. In this regard, communications module 240 may be in communicationwith processor 210, such as via a bus. Communications module 240 mayinclude, for example, an antenna, a transmitter, a receiver, atransceiver, network interface card and/or supporting hardware and/orfirmware/software for enabling communications with another computingdevice. Communications module 240 may be configured to receive and/ortransmit any data that may be stored by memory 220 using any protocolthat may be used for communications between computing devices.Communications module 240 may additionally or alternatively be incommunication with the memory 220, input/output module 250 and/or anyother component of circuitry 200, such as via a bus.

Input/output module 250 may be in communication with processor 210 toreceive an indication of a user input and/or to provide an audible,visual, mechanical, or other output to a user (e.g., employee and/orworker). Some example visual outputs that may be provided to a user bycircuitry 200 are discussed in connection with FIGS. 1-10 . Inaccordance with an aspect of the present disclosure, the input/outputmodule 250 may request an input from a user and/or a worker. Inresponse, the worker may provide a voice command. The term “voicecommand” as used herein may refer to audio and/or speech inputs receivedfrom a user as part of a workflow task or audio and/or speech outputsprovided to a user. For example, in a palletizing application, theinput/output module 250, such as a speaker and/or a display, may requesta user to provide a pallet size needed for creating a palletizingpattern. In response, the worker may provide a voice command through theinput/output module 250, such as a microphone. Additionally and/oralternatively, the worker may provide input through button commandsmanually inputted by the worker. The input/output module 250, theprocessor 210, and/or the workflow system 400 may additionally record atime of receipt of the input from the worker. Further, the voicecommands and/or the button commands may be processed by the processor210 and/or the workflow system 400 to create corresponding textcommands. The term “text command” as used herein may refer to digitaldata created by converting the voice commands to points in data storedin a database. Specifically, the processor 210 and/or any othercircuitry may digitize received voice command, and then may performprocessing, such as, sampling, filtering, recognition, and the like, tomatch keywords and/or data points in the voice command with data storedin a database, to generate a text command corresponding to the voicecommand. FIG. 8 provides further details on the processing of voicecommand, as will be described later.

The processor 210 and/or the workflow system 400 may further create adialogue-stream based on the text command and corresponding time ofreceipt of the inputs from the worker. The term “dialogue-stream” asused herein may refer to one or more time-stamped interactions betweenthe worker and the associated device. The dialogue-stream may includeall the text commands converted from voice commands, along with the timeof receipt of the voice command. For example, the worker may provide avoice command input to the request for providing pallet size, as, “800by 600 by 163.” This voice command may be time-stamped and stored as adialogue stream for further processing.

As such, input/output module 250 may include support, for example, for akeyboard, a mouse, a joystick, a display, a touch screen display, amicrophone, a speaker, a RFID reader, barcode reader, biometric scanner,and/or other input/output mechanisms. In embodiments whereininput/output module 250 is embodied as a server or database, aspects ofinput/output module 250 may be reduced as compared to embodiments whereinput/output module 250 is implemented in an end-user machine (e.g.,operator device and/or supervisor device) or other type of devicesdesigned for complex user interactions. In some embodiments (like othercomponents discussed herein), input/output module 250 may even beeliminated from circuitry 200. Alternatively, such as in embodimentswherein some aspects of circuitry 200 is embodied as a server ordatabase, at least some aspects of input/output module 250 may beembodied on an apparatus used by a user that is in communication withthe rest of circuitry 200. Input/output module 250 may be incommunication with the memory 220, communications module 240, and/or anyother component(s), such as via a bus. One or more than one input/outputmodule and/or other component can be included in circuitry 200.

The workflow database 300 and the workflow system 400 may also orinstead be included and configured to perform the functionalitydiscussed herein related to workflow performance. In some embodiments,some or all of the functionality of generating information for workflowperformance may be performed by processor 210. In this regard, theexample processes and algorithms discussed herein can be performed by atleast one processor 210, workflow database 300, and/or workflow system400. For example, non-transitory computer readable media can beconfigured to store firmware, one or more application programs, and/orother software, which include instructions and other computer-readableprogram code portions that can be executed to control each processor(e.g., processor 210, workflow database 300, and/or workflow system 400)of the components of circuitry 200 to implement various operations,including the examples shown above. As such, a series ofcomputer-readable program code portions are embodied in one or morecomputer program goods and can be used, with a computing device, server,and/or other programmable apparatus, to produce machine-implementedprocesses.

In some embodiments, a workflow database 300 may be provided thatincludes various relevant information for the workflow performancesystem. For instance, as shown in FIG. 3 , in this embodiment, theworkflow database 300 includes employee data 311, task data 312,employee historical performance data 313, and company performance data314. Various other data may be included in the workflow database 300. Asadditional tasks are performed, the system 200 may receive additionalinformation regarding the tasks and the employee (e.g., performanceinformation), which may be stored in the workflow database 300. Inaddition, as additional information is developed regarding productsand/or services provided by the company, the system 200 may receive thisadditional information and store the information in the workflowdatabase 300 with distinct tasks for handling (e.g., manufacturing orprocessing) such products or services for future use. Additionally oralternatively, the workflow database 300 may include analytical enginedata 315 which provides any additional information needed by theworkflow system 400 in analyzing inputs and requests and generating theappropriate response.

For example, workflow system 400 can be configured to analyze multiplesets of data (e.g., including various combinations of employee data,task data, employee historical performance data, company data, etc.),such as the data in the workflow database 300. In this way, the workflowsystem 400 may support multiple algorithms, including those discussedbelow with respect to employee data, task data, employee historicalperformance data, company data, etc., so that the selected algorithm maybe chosen at runtime. Further, the present configuration can enableflexibility in terms of configuring additional contexts.

In some embodiments, with reference to FIG. 4 , the workflow system 400may include a context determination module 420, analytical engine 430,and communications interface 440, all of which may be in communicationwith the workflow database 300. The workflow system 400 may receive oneor more inputs or requests (e.g., voice commands) and may generate anappropriate response. The workflow system 400 may use any of thealgorithms or processes disclosed herein for receiving a request/inputand generating a response. In some other embodiments, such as when thecircuitry 200 is embodied in a server 160, supervisor device 150, and/oroperator devices 110A-110N, the workflow system 400 may be located inanother circuitry 200 or another device, such as another server 160,supervisor device 150, and/or operator devices 110A-110N.

The workflow system 400 can be configured to access data correspondingto one or more employees and products/services (by way of tasks), andgenerate one or more responses and/or indications. In accordance with anaspect of the present disclosure, the workflow system 400 and/or theprocessor 210 may receive one or more voice commands and prepare anassociated dialogue-stream, as described above. A dialogue stream mayinclude device and/or system initiated events and user inputs and/oractions, referred to here as ‘user-initiated events.’ The workflowsystem 400 and/or the processor 210 may filter the dialogue-stream foruser-initiated events. Further, a workflow record may be created fromthe dialogue-stream based on the associated tasks and sub-tasks. In anembodiment, the workflow record may be analyzed by the workflow system400 and/or the processor 210 to monitor worker efficiency, such as, butnot limited to, duration of tasks and/or sub-tasks, rate of interactionwith the device, duration of a task, and the like. The term “duration ofa task” may refer to time between the start of a task and the end of thetask in the database. Further, “average duration” may refer to anaverage time spent on a task by one or more workers. In someembodiments, the method may further include determining an averageduration of tasks associated with the voice commands by averaging theduration of each task over a stastically significant number ofrepetitions of the task. The average duration may be a weighted average,arithmetic mean, geometric mean, harmonic mean, quadratic mean, mode,median, etc. Centroids and clustering algorithms may also be used. Asused herein, “average duration” is generally used to refer to a durationcalculated based on more than one duration and may be used, in someembodiments, to determine a “set duration” or baseline for comparing theduration of one or more tasks. Additional calculations may be made andincorporated into determining the set duration or baseline for comparingdurations of tasks. For instance, other statistical information (e.g.,standard deviation, etc) may be calculated. The workflow system 400and/or the processor 210 may also monitor a time spent between adjacenttasks. The term “adjacent task(s)” may refer to two tasks in a workflowsolution, such that one of the adjacent tasks is performed prior to theother adjacent task with no intervening task. Further, if one or more ofthe worker efficiency parameters are outside of the parameters orthresholds set to define an efficient worker, a supervisor and/or amanager may be notified of the inefficient worker. The parameters orthresholds set to define an efficient worker may be respective to theefficiency parameters and may be determined prior to start of the taskor may be a running calculation based on the task and/or other tasks.Additionally and/or alternatively, a supervisor device may receive thedialogue-stream and/or the workflow record for monitoring efficiency andproductivity of workers.

With reference to FIG. 4 , whether used locally or over a network, theworkflow system 400 may be used to analyze the workflow performance,identify tasks and subtasks, create alarms associated with the tasks andsubtasks, and notify supervisors or management as to the relevanttasks/subtasks or employee's work performance. The system may receive aplurality of inputs 410, 415 from the circuitry 200 (e.g., voicecommands or text commands) and process the inputs within the workflowsystem 400 to produce an output 450 (e.g., dialogue-stream, workflowrecord, alarms, alerts, etc.). In some embodiments, the workflow system400 may execute context determination module 420, process the data in ananalytical engine 430, and output the results via the communicationsinterface 440. Each of these steps may pull data from a plurality ofsources including the workflow database 300.

When inputs 410, 415 are received by the workflow system 400, a contextdetermination using context determination module 420 may first be made.A context determination includes such information as employee profiledata (e.g., what employee is associated with the input 410, 415),employee historical performance data (e.g., how has the employee handledtasks associated with the input 410, 415 in the past), task data (e.g.,what task is associated with the input 410, 415), preference data of thesystem, and what request or indication was received as the input 410,415. These inputs may give context to the workflow system's 400 analysisto determine the output. For example, the context determination module420 may inform the workflow system 400 as to what action is appropriate.

The workflow system 400 may then analyze the inputs 410, 415 using theanalytical engine 430. The analytical engine 430 draws information aboutthe task/employee/product/company etc. from the workflow database 300and then, in light of the context determination module's 420determination, computes an appropriate response.

As will be appreciated, any such computer program instructions and/orother type of code may be loaded onto a computer, processor or otherprogrammable apparatus's circuitry to produce a machine, such that thecomputer, processor other programmable circuitry that execute the codeon the machine create the means for implementing various functions,including those described herein.

It is also noted that all or some of the information presented by theexample displays discussed herein can be based on data that is received,generated and/or maintained by one or more components of a local ornetworked system and/or circuitry 200. Data may be received, generated,and/or maintained on an ongoing basis to provided continuously updatedcalculations for analyzing the workers' efficiency. In some embodiments,one or more external systems (such as a remote cloud computing and/ordata storage system) may also be leveraged to provide at least some ofthe functionality discussed herein.

As described above and as will be appreciated based on this disclosure,embodiments of the present invention may be configured as methods,personal computers, servers, mobile devices, backend network devices,and the like. Accordingly, embodiments may comprise various meansincluding entirely of hardware or any combination of software andhardware. Furthermore, embodiments may take the form of a computerprogram product on at least one non-transitory computer-readable storagemedium having computer-readable program instructions (e.g., computersoftware) embodied in the storage medium. Any suitable computer-readablestorage medium may be utilized including non-transitory hard disks,CD-ROMs, flash memory, optical storage devices, or magnetic storagedevices.

Embodiments of the present invention have been described above withreference to block diagrams and flowchart illustrations of methods,apparatuses, systems and computer program goods. It will be understoodthat each block of the circuit diagrams and process flowcharts, andcombinations of blocks in the circuit diagrams and process flowcharts,respectively, can be implemented by various means including computerprogram instructions. These computer program instructions may be loadedonto a general purpose computer, special purpose computer, or otherprogrammable data processing apparatus, such as processor 210, workflowdatabase 300, and/or workflow system 400 discussed above with referenceto FIG. 2 , to produce a machine, such that the computer program productincludes the instructions which execute on the computer or otherprogrammable data processing apparatus create a means for implementingthe functions specified in the flowchart block or blocks.

These computer program instructions may also be stored in acomputer-readable storage device (e.g., memory 220) that can direct acomputer or other programmable data processing apparatus to function ina particular manner, such that the instructions stored in thecomputer-readable storage device produce an article of manufactureincluding computer-readable instructions for implementing the functiondiscussed herein. The computer program instructions may also be loadedonto a computer or other programmable data processing apparatus to causea series of operational steps to be performed on the computer or otherprogrammable apparatus to produce a computer-implemented process suchthat the instructions that execute on the computer or other programmableapparatus provide steps for implementing the functions discussed herein.Accordingly, blocks of the block diagrams and flowchart illustrationssupport combinations of means for performing the specified functions,combinations of steps for performing the specified functions and programinstruction means for performing the specified functions. It will alsobe understood that each block of the circuit diagrams and processflowcharts, and combinations of blocks in the circuit diagrams andprocess flowcharts, can be implemented by special purpose hardware-basedcomputer systems that perform the specified functions or steps, orcombinations of special purpose hardware and computer instructions.

The operator devices 110A-110N, supervisor device 150, and/or server 160may include one or more input devices, including but not limited to akeyboard, a number pad, a mouse, a touch-sensitive display, a navigationkey, a function key, a microphone, a voice recognition component, anyother mechanism capable of receiving an input from a user, or anycombination thereof. Further, the operator devices 110A-110N, supervisordevice 150, and/or server 160 may include one or more output devices,including but not limited to a display, a speaker, a haptic feedbackmechanism, a printer, lights, any other mechanism capable of presentingan output to a user, or any combination thereof.

The operator devices 110A-110N, supervisor device 150, and/or server 160may include components for monitoring and/or collecting informationregarding the user or external environment in which the component isplaced. For instance, the operator devices 110A-110N, supervisor device150, and/or server 160 may include sensors, scanners, and/or othermonitoring components. In some embodiments, scanners may be used todetermine the presence of certain individuals or items. For example, insome embodiments, the components may include a scanner, such as anoptical scanner, RFID scanner, and/or other scanner configured to readhuman and/or machine readable indicia physically associated with anitem.

FIG. 5 illustrates an exemplary user device (e.g., operator devices110A-110N, supervisor device 150, and/or server 160) according to anexample embodiment. In the embodiment illustrated in FIG. 5 , the userdevice 500 is a headset that includes a wireless enabled voicerecognition device that utilizes a hands-free profile.

In one embodiment, the headset may be substantially similar to theheadset disclosed in U.S. Provisional Patent Application No. 62/097,480filed Dec. 29, 2014, U.S. Provisional Patent Application No. 62/101,568,filed Jan. 9, 2015, and U.S. patent application Ser. No. 14/918,969, andthe disclosures therein are hereby incorporated by reference in theirentireties.

In the embodiment illustrated in FIG. 5 , the user device 500 includesan electronic module 502. In this embodiment, elements are incorporatedinto an electronics module 502 rather than the headset 503, to provide along battery life consistent with long work shifts. One or morecomponents of circuitry 200 may be incorporated in the electronic module502 and/or the headset 503. The electronics module 502 is remotelycoupled to a light-weight and comfortable headset 503 secured to aworker head via a headband 504. The headband 504 can be a band that isdesigned to fit on a worker's head, in an ear, over an ear, or otherwisedesigned to support the headset. The headset 503 includes one or morespeakers 505 and includes one or more microphones. For instance, in theembodiment illustrated in FIG. 5 , the headset 503 includes microphones506, 507. Microphone 507 can provide for noise cancellation bycontinuously listening to and blocking environmental sounds to enhancevoice recognition and optionally provide for noise cancellation. In someembodiments (not shown), the electronics module 502 can be integratedinto the headset 503 rather than being remotely coupled to the headset503. Various configurations may be used without deviating from theintent of the present disclosure.

In some embodiments, the electronics module 502 can be used to offloadseveral components of the headset 503 to reduce the weight of theheadset 503. In some embodiments, one or more of a rechargeable or longlife battery, display, keypad, Bluetooth® antenna, and printed circuitboard assembly (PCBA) electronics can be included in the electronicsmodule 502 and/or otherwise incorporated into the user device 500.

In the embodiment illustrated in FIG. 5 , the headset 503 attaches tothe electronics module 502 via a communication link such as a smallaudio cable 508, but could instead communicate with the electronicsmodule 502 via a wireless link. In some embodiments, the headset 503 hasa low profile. For instance, headset 503 can be minimalistic inappearance in some embodiments, such as a Bluetooth earpiece/headphone.

Electronics module 502 can be used with various headsets 503, such asVOCOLLECT™ headsets. The electronics module 502 can read a uniqueidentifier (I.D.) of the headset 503, which can be stored in thecircuitry of the user device 500 (e.g., the circuitry 200) and is alsoused to electronically couple the speakers and microphones toelectronics module 502. In one embodiment, the audio cable 508 includesmultiple conductors or communication lines for signals which can includea speaker+, speaker−, ground digital, microphone, secondary microphone,and microphone ground. The electronics module 502 can utilize a userconfigurable attachment 509, such as a plastic loop, to attach to auser. For instance, in the embodiment illustrated in FIG. 5 , theelectronics module 502 can be mounted to a worker torso via a lapel clipand/or lanyard.

In some embodiments, the headset 503 can include a small lightweightbattery, such as when a wireless link between the headset 503 andelectronics module 502 is used, such as Bluetooth type of communicationlink. The communication link can provide wireless signals suitable forexchanging voice communications.

In some embodiments, voice templates can be stored locally in theelectronic module 502 and/or the headset 503 as part of the circuitry200 to recognize a user's voice interactions and may convert theinteraction into text based data and commands for interaction with anapplication running in the circuitry 200. For example, the user device500 can perform voice recognition in one embodiment utilizing the voicetemplates. Further, in one embodiment, the first stages of voicerecognition can be performed in the user device 500, with further stagesperformed on a server 160. In further embodiments, raw audio can betransmitted from user device 500 to the server 160 where the finalstages of voice recognition are completed.

FIG. 6 illustrates an exemplary block diagram of an electronics module502 in accordance with some embodiments of the present disclosure. Thecomponents illustrated in FIG. 6 may be in addition to one or morecomponents of the circuitry 200 shown in FIG. 2 , which may be part ofthe electronics module 502. In some embodiments, one or more of thecomponents illustrated in FIG. 6 may be included in the electronicsmodule 502 and/or other parts of the user device 500, operator devices110A-110N, supervisor device 150, and/or server 160.

In the embodiment shown in FIG. 6 , the electronics module 502 includesan enclosure, such as a plastic case, with a connector 510 that mateswith a complimentary mating connector (not shown) on audio cable 508. Aninternal path 511 is used to communicate between multiple componentswithin the electronics module 502 enclosure. In one embodiment, an inputspeech pre-processor (ISPP) 512 converts input speech into pre-processedspeech feature data. An input speech encoder (ISENC) 513 encodes inputspeech for transmission to one or more other parts of circuitry 200 forreconstruction and playback and/or recording. A raw input audio samplepacket formatter 514 transmits the raw input audio to one or more otherparts of circuitry 200 using an application-layer protocol to facilitatecommunications between the voice terminal and headset 503 as thetransport mechanism. For the purposes of the transport mechanism, theformatter 514 can be abstracted to a codec type referred to as InputAudio Sample Data (IASD). An output audio decoder (OADEC) 515 decodesencoded output speech and audio for playback in the headset 503. A rawoutput audio sample packet reader 516 operates to receive raw audiopackets from one or more other parts of circuitry 200 using thetransport mechanism. For the purposes of the transport mechanism, thisformatter 514 can be abstracted to a codec type referred to as OutputAudio Sample Data (OASD). A command processor 517 adjusts the headset502 hardware (e.g., input hardware gain level) under control of one ormore other parts of circuitry 200. A query processor 518 allows one ormore other parts of circuitry 200 to retrieve information regardingheadset operational status and configuration. Path 511 is also coupledto network circuitry 519 to communicate via wired or wireless protocolwith one or more other parts of circuitry 200. The ISPP 512, ISENC 513,and raw input audio formatter 514 are sources of communication packetsused in the transport mechanism; the OADEC 515 and raw output audioreader 516 are packet sinks. The command and query processors 517, 518are both packet sinks as well as sources (in general they generateacknowledgement or response packets).

FIG. 7 illustrates an exemplary user device (e.g., operator devices110A-110N, supervisor device 150, and/or server 160) according to anexample embodiment. In the embodiment illustrated in FIG. 7 , the userdevice is a handset 704 (e.g., a mobile device or tablet device). Thehandset 704 may include one or more components of circuitry 200 asexplained with regards to FIG. 1 and may include one or more of thecomponents discussed with regards to the headset of FIGS. 5 and 6 (e.g.,voice templates, speech encoders, etc.). The handset 704 may include oneor more microphones 706 and one or more speakers 708, which may beconnected to a set of headphones. The handset 704 can also include oneor more antenna. The microphone 706 receives speech or sound andtransmits the received speech and sound to one or more components ofcircuitry 200 in the handset 704. The speakers 708 receive an audiotransmission from one or more components of circuitry 200 in the handset704 and output the audio transmission in the form of speech or sound. Inan embodiment, the speakers 708 can also include noise cancellation. Thehandset 704 may connect with one or more other operator devices110A-110N, supervisor device 150, and/or server 160 as explained withregards to FIG. 1 . For instance, in some embodiments, the handset 204may connect to a wireless headphone via a Bluetooth connection, wherethe wireless headphone includes a microphone and speaker for receivingspeech and outputting speech or sound. The handset 204 can also includea user input device and output device (such as the display 203 formingan interface) to send and receive additional non-auditory informationfrom circuitry 200, whether incorporated into the handset 704 or inother operator devices 110A-110N, supervisor device 150, and/or server160. The display 203 of FIG. 7 may be a backlit LCD or OLED display.With the use of a handset 704 having one or more microphones 706 and oneor more speakers 708, a user can communicate with a central server(e.g., server 160) and/or with other user devices (e.g., operatordevices 110A-110N, supervisor device 150, and/or server 160).

Although FIG. 7 illustrates one example of a handheld device, variouschanges may be made to FIG. 7 . For example, all or portions of FIG. 7may represent or be included in other handheld devices and may be usedin conjunction with a headset such as the headset of FIG. 5 . Also, thefunctional division shown in FIG. 7 is for illustration only. Variouscomponents could be combined, subdivided, or omitted and additionalcomponents could be added according to particular needs.

One suitable device for implementing the present disclosure may be theTALKMAN® product available from VOCOLLECT™ of Pittsburgh, Pa. Inaccordance with one aspect of the present disclosure, the user device(e.g., operator devices 110A-110N, supervisor device 150, and/or server160) may be used in a voice-driven system, which may use speechrecognition technology for communication. In an embodiment, the userdevice may provide hands-free voice communication between the user andthe device. To that end, digital information may be converted to anaudio format, and vice versa, to provide speech communication betweenthe user device or an associated system and the user. In an exampleembodiment, the user device may receive digital instructions from acentral computer and/or a server and may convert those instructions toaudio to be heard by the user. The user may then reply, in a spokenlanguage, and the audio reply or the speech input may be converted to auseable digital format to be transferred back to the central computerand/or the server. In other embodiments, the user device may operateindependently, in an offline mode, such that speech digitization,recognition and/or synthesis for implementing a voice-driven workflowsolution may be performed by the user device itself.

FIG. 8 illustrates an exemplary embodiment of a method 800 for providingvoice based communication and/or speech dialogue between a user and auser device in a workflow environment. The method 800 may includegenerating a voice command for a user 802. In an embodiment, the userdevice may include output devices, such as, speakers for receivingdigital instructions and/or commands from one or more components of theelectronics module 502 in the user device and output the audiotransmission in the form of speech or sound. In an embodiment, thecircuitry 200 may include a dialogue engine in operative communicationwith one or more components of the circuitry 200. For instance, thedialogue engine may be stored in the memory. For a given workflow task,the user device may be required to carry on a dialogue with the user,which may include a series of messages or instructions or questions tooutput to the user. Such elements of the dialogue may be referred to as“prompts”. In accordance with one aspect of the present disclosure, theprompts may be output in the form of voice commands generated by thedialogue engine and/or the speakers. For example, a prompt asking a userfor a quantity of goods may designate that an audible prompt or a voicecommand be provided by the speaker to the user, in accordance with theinvention. Additionally, the voice command may also include informationindicating which input modes or components are appropriate to capture orreceive any incoming data for answering or addressing the prompt and/orthe voice command.

As described above, the dialogue engine may be instructed by one or morecomponents of the circuitry 200 as to which modes or components to usefor any given prompt. Thus, the dialogue engine may control theexecution of the workflow task by managing the output components and theinput components of the user device in conjunction with the workflowsystem 400. For example, the dialogue engine may provide an outputprompt by causing a voice synthesizer with a text-to-speech (TTS)functionality to produce an appropriate voice command played by thespeaker.

The method 800 may further include receiving a speech input from a userin response to the voice command 804. In accordance with one aspect ofthe present disclosure, the workflow task being executed may include aseries of instances or junctures where an input is expected to bereceived in response to the voice command from the user. For example, avoice command asking a user for a quantity of goods may require a userto provide an input, such as, a voice command (e.g., speech input),providing quantity information, in accordance with the invention. In anexample embodiment, the user device, as described above, may furtherinclude input devices, such as a microphone for receiving speech inputsfrom a user. The microphone may further transmit the received speechinput to one or more components of circuitry 200 for further processingand recognition.

The method 800 may include digitizing the received speech input andprocessing digitized speech 806. In accordance with one aspect of thepresent disclosure, a microphone or other electro-acoustical componentsof the user device may receive a speech input from a user and mayconvert the speech input into an analog voltage signal. The analogvoltage signal may then be forwarded to one or more components of thecircuitry 200, such as, but not limited to, analog to digitalconvertors, audio processing circuits such as audio filters, correlationelectronics module, and the like. Specifically, the circuitry 200 mayinclude suitable digitization electronics module (e.g., in electronicsmodule 502) for providing appropriate representations of the speechinput received from the user for further processing. In an embodiment,the digitization electronics module may include an audio coder/decoderchip or CODEC. Further, the digitization electronics module may includenecessary filters, equalization electronics module and/or software forconverting the analog speech input into a digitized stream of data thatcan be separated into separate units for analysis.

Further, in accordance with one aspect of the present invention, thecircuitry 200 may further include audio and/or speech analysiselectronics module and signal processing electronics module foranalyzing and further processing the digitized speech inputs. Forexample, the user device may include spectral transformation electronicsmodule, or audio filtering electronics module, such as Mel scalefilters, which may create a spectral transform of the digitized audiosignals and may provide a series of sampled representations or valuesassociated with the digitized audio signals. In an example embodiment,the circuitry 200 may divide the digital stream of data that may becreated into a sequence of time-slices, or frames, each of which maythen be processed by a feature generator, thereby producing features(vector, matrix, or otherwise organized set of numbers representing theacoustic features of the frames). The features may then be used byspeech recognition components of the circuitry 200 for furtherprocessing.

The method 800 may further include performing speech recognition tomatch speech input to an expected response 808. In accordance with oneaspect of the present disclosure, a speech recognition search algorithmfunction, realized by an appropriate circuit and/or software in the userdevice may analyze the features, as described above, to determine whathypothesis to assign to the speech input captured by the microphone ofthe device. As is known in the art, in one recognition algorithm, therecognition search relies on probabilistic models provided through adatabase of suitable models to recognize the speech input. Each of themodels in the database may either be customized to a user or be genericto a set of users.

One common modeling technique utilized for speech recognition includesHidden Markov Models (HMM). In speech recognition, these models may usesequences of states to describe vocabulary items, which may be words,phrases, or subword units. As used herein, the term “word” may refer toa vocabulary item, and thus may mean a word, a segment or part of aword, or a compound word, such as “next slot” or “say again.” Therefore,the term “word” may not be limited to just a single word. Each state inan HMM may represent one or more acoustic events and may serve to assigna probability to each observed feature vector. Accordingly, a paththrough the HMM states may produce a probabilistic indication of aseries of acoustic feature vectors. The model may be searched such thatdifferent, competing hypotheses (or paths) are scored; a process knownas acoustic matching or acoustic searching. A state S may be reached ata time T via a number of different paths. For each path reaching aparticular state at a particular time, a path probability may becalculated. Using the Viterbi algorithm, each path through the HMM maybe assigned a probability. In particular, the best path may be assigneda probability. Furthermore, each word in the best path may be assigned aprobability. Each of these probabilities may be used as a confidencefactor or combined with other measurements, estimates or numbers toderive a confidence factor. The path with the highest confidence factor,the best hypothesis, can then be further analyzed.

When in operation, the search algorithm (which can be implemented usingHidden Markov Models with a Viterbi algorithm or other modelingtechniques such as template matching dynamic time warping (DTW) orneural networks), in essence, may compare the features generated, asdescribed above, with reference representations of speech, or speechmodels, in a database in order to determine the word or words that bestmatch the speech input. In an embodiment, part of this recognitionprocess may be to assign a confidence factor for the speech to indicatehow closely the sequence of features from the search algorithm matchesthe closest or best-matching models in the database. As such, ahypothesis consisting of one or more vocabulary items and associatedconfidence factors may be directed to an acceptance algorithm todetermine expected response. In accordance with the above embodiment, ifthe confidence factor is above a predetermined acceptance threshold,then the acceptance algorithm may decide to accept the hypothesis asrecognized speech. If, however, the confidence factor is not above theacceptance threshold, as utilized by the acceptance algorithm, then theacceptance algorithm may decide to ignore or reject the recognizedspeech. The user device may then prompt the user to repeat the speechinput. In this instance, the user may repeat the speech input providedto the microphone.

The method 800 may further include executing the workflow task based onthe speech input 810. In some embodiments, a user may be executing atask in a workflow solution, thereby receiving and providing speechinputs to the microphone of the user device. The speech inputs receivedfrom a user may relate to a voice command and/or query related to thetask in the workflow solution. For example, the user device may providea voice command to the user to move to a pick-up location correspondingto an item to be picked up in a warehouse and/or a distributionfacility. Once the user arrives at the pick-up location and picks thearticle, the user may be required to provide a speech input indicating apick-up confirmation to the user device to move further in the workflow.In some embodiments, a user may provide a speech input indicating anidentification number, or the like, of an item picked-up by the user,aisle and/or shelf number confirming a pick-up location, and the like.Thus, when the digitized and processed speech input matches the expectedresponse, the workflow task is executed by the circuitry 200 and/orassociated processor based on the speech input.

FIG. 9 illustrates an exemplary embodiment of a method 900 foridentifying inefficient workers in a workplace. The operationsillustrated in FIG. 9 may, for example, be performed by, with theassistance of, and/or under the control of the server 160, such asthrough the use of circuitry 200. Alternatively, the operationsillustrated in FIG. 10 may, for example, be performed by, with theassistance of, and/or under the control of an operator device 110 and/orsupervisor device, such as through the use of circuitry 200. Variousaspects of the method 900 may be performed by one or more of theoperator device 110, supervisor device 150, and/or server 160.

In accordance with an aspect of the present disclosure, the method 900may include receiving a dialogue-stream from a user device 902. Asdescribed above, a dialogue-stream may include text commands and a timeassociated with each text command. Further, the text commands may beformed by transforming audio signals associated with voice commandsreceived from a user to one or more points in data that are stored in adatabase. The database may be stored in a memory of the operator device110, supervisor device 150, server 160, and/or may be remotely storedand may be accessible to the above-mentioned devices. The database mayinclude a list of voice templates and/or keywords and theircorresponding data points in text. Thus, the dialogue-stream may includetimestamp of receipt of each voice command and text commands for eachvoice command. The text commands and time of receipt of each voicecommand associated with the respective text command may be gathered intoa single collection organized by the time of receipt.

In some embodiments, the user device 110, supervisor device 150, server160, and/or any other associated device may further recognize tags inthe dialogue-stream. The tags may indicate that an associated textcommand in the dialogue-stream originated from a user as opposed to theuser device (and is thus “user-initiated”). Further, the user device110, supervisor device 150, server 160, and/or any other associateddevice may further remove items in the dialogue-stream that are notassociated with the tags to create a user-initiated dialogue-stream. Theuser-initiated dialogue-stream dialogue stream may be analyzed furtherfor monitoring efficiency of a worker.

The method 900 may further include creating a workflow record for theparticular user by identifying a task associated with a voice commandfrom a user based on each text command associated with the respectivevoice command 904 and determining a duration of the task 906. Inaccordance with one aspect of the present disclosure, the user device110, supervisor device 150, server 160, and/or any other associateddevice may create a workflow record for a particular user based onidentifying a task associated with a voice command from a user based oneach text command associated with the respective voice command anddetermining a duration of the task. For example, the workflow record mayinclude information, such as, User A/Task—Picking Complete/Duration—4minutes, User B/Task—Picking Complete/Duration—10 minutes, etc. Theworkflow record may be used to monitor the efficiency of each worker. Anexemplary workflow record is shown below:

event duration event start in timestamp Event Generated time stampseconds comment 1501605714 ==> Welcome to the s9873216 system. Say readylogin begins 1501605720 <== ready 1501605726 ==> Password? 1501605725<== * 1501605728 <== * 1501605733 <== * 1501605734 LogInComplete1501605714 21 1501605736 ==> Region? 1501605737 <== 1 1501605739 ==>stockless, correct? 1501605742 <== yes SelectRegionComplete 1501605736 6 1501605744 ==> To receive work, say ready 1501605749 <== ready ==>means device speaks <== means operator speaks

In accordance with one aspect of the present disclosure, identifying atask associated with a voice command may include determining a currentstate of the user device, determining a next state of the user device,identifying a trigger to move to the next state of the user device, andadding the task to the workflow record after identifying the trigger. Insome embodiments, the current state may be determined by the earliestreceived voice command and associated text command or a voice commandreceived from the user device that has not been associated with anytask. The current state may relate to the software model of the userdevice as well as the actual state of the device (on, off, sleeping,charging, on-air. The next state may be determined by identifying a taskintended to follow the current state from the database or anotherdatabase. Further, the trigger may include a dialogue of interestrelated to the next state and may be provided in any input form such asa voice command or button command. The dialogue of interest may bematched to specific patterns and that match is the trigger. The specificpatterns may be respective to the particular action/task being performed(e.g., “yes” may only change the state of the user device when the useris engaged in responding to the device's request for confirmation). Aworkflow record may be formed of multiple individual tasks, each havinga start, a trigger, and an end. Each task may have further sub-tasks.For example, a workflow record for a picking operation, may includetasks such as user login, receiving pick list, routing, providing firstpick location to worker, confirming pick operation, providing next picklocation to user, etc. Furthermore, a task, such as user login, may haveone or more sub-tasks, such as receiving username and/or user ID fromworker, receiving password from worker, authenticating, and the like. Atask may have multiple pick and travel actions. A new task or particularother defined task may end the current task when it starts. Each taskmay have an associated trigger to end a current task and move theworkflow record to the adjacent task. In some embodiments, a task may beended when a new task begins, or it may continue if it is determined bythe logic that the new task may be part of the previous task. Theoriginal task duration may or may not continue to accrue, depending onthe nature of the new task (e.g. acquiring a new pallet while performinga pick should not cause the duration of the pick to accrue, but the timespent before and after the pallet acquisition may).

In some embodiments, button commands may be manually inputted by theworker in the workplace. The processor may incorporate the buttoncommands into the text commands by identifying one or more points indata stored in the database or another database related to each of thebutton commands. The processor may further incorporate the text commandsfrom the button commands into the dialogue-stream.

The method 900 may further include determining a duration of the task906. In accordance with one aspect of the present disclosure, the userdevice 110, supervisor device 150, server 160, and/or any otherassociated device may determine the duration of a task may includeidentifying a text command or a voice command received from the userdevice in the dialogue-stream associated with a start of the task in thedatabase or another database, identify a text command or a voice commandreceived from the user device in the dialogue-stream associated with anend of the task in the database or another database, and determine atime between the start of the task and the end of the task. In someembodiments, the method may further include determining an averageduration of tasks associated with the voice commands by averaging theduration of each task over a statistically significant number ofrepetitions of the task.

In some embodiments, the user device 110, supervisor device 150, server160, and/or any other associated device may further determine a rate ofinteraction of a user by calculating a number of text commands over aperiod of time in the user-initiated dialogue-stream. In someembodiments, the method 900 may further include creating an alarmassociated with the task. The alarm may notify a supervisor device thatthe particular user has exceeded a maximum duration of the task bytransmitting an indication to the supervisor device 150 when theduration of the task exceeds the maximum duration of the task in theworkflow record. For example, if the maximum duration set for thepicking operation, as described above, is 7 minutes, an alarm and/or anindication may be sent to the supervisor device 150 when User B exceedsthe 7-minute time duration. In further embodiments, the workflow recordand/or the dialogue-stream may be used to identify periods ofnon-activity of the worker with the device.

In some embodiments, the method 900 may further include receivingdialogue-streams from multiple user devices. Each dialogue-stream mayinclude text commands and a time associated with each text command. Thetext commands may be formed by transforming voice commands into the textcommands by converting audio signals associated with the voice commandsto one or more points in data that are stored in the database or anotherdatabase. The method may further include creating workflow records forthe users by identifying a task for a voice command in each workflowrecord and determining a duration of each task of each workflow record,and comparing the duration of each task of each workflow record. In someembodiments, the method 900 may further include filtering the workflowrecords for a single task, and determining an average duration of thesingle task over the workflow records, a minimum duration of the singletask over the workflow records, a maximum duration of the single taskover the workflow records, and other statistical information, such asstandard deviation, correlation, etc. Other calculations may bedetermined and used herein. In some embodiments, the method 900 mayfurther include creating an alarm associated with the single task. Thealarm may notify a supervisor device that a user has exceeded a setduration of the single task by transmitting an indication to thesupervisor device when the duration of the single task exceeds the setduration in the workflow record of the user. In some embodiments, thealarm may be transmitted to the user device. In some embodiments, themethod 900 may further include identifying a period of non-activity of auser by identifying a period of time where the rate of interactiondecreases to a defined value and causing an indication to be transmittedto a supervisor device notifying a supervisor of the particular user ofthe occurrence of the period of non-activity.

FIG. 10 illustrates an exemplary embodiment of a method 1000 foridentifying inefficient workers in a workplace. The operationsillustrated in FIG. 10 may, for example, be performed by, with theassistance of, and/or under the control of the supervisor device 150,such as through the use of circuitry 200. Alternatively, the operationsillustrated in FIG. 10 may, for example, be performed by, with theassistance of, and/or under the control of operator device 100 and/orserver 160. Various aspects of the method 1000 may be performed by oneor more of the operator device 110, supervisor device 150, and/or server160.

In accordance with an aspect of the present disclosure, the method 1000may include receiving a workflow record from a user device 110 or aserver 160 1002. As described above, the supervisor device 150 mayreceive a workflow record for monitoring productivity and/or efficiencyof a worker. The workflow record may be created by the user device 110,server 160, or any other associated device based on identifying a taskassociated with a voice command from a user based on each text commandassociated with the respective voice command and determining a durationof the task. Further, the workflow record may be used to generate anefficiency metric. For example, the workflow record may includeinformation for the supervisor to determine a rate of interaction of auser by calculating a number of text commands over a period of time inthe user-initiated dialogue-stream, a maximum duration of the task by auser, etc. In further embodiments, the workflow record and/or thedialogue-stream may be used to identify periods of non-activity of theworker with the device by the supervisor. Further, as described above,dialogue-stream may include text commands and a time associated witheach text command. The text commands may be created by transformingaudio signals associated with voice commands received from a user to oneor more points in data that are stored in a database. The database maybe stored in a memory of the device 110, supervisor device 150, server160, and/or may be remotely stored and may be accessible to theabove-mentioned devices. The database may include a list of voicetemplates and/or keywords and their corresponding data points in text.Thus, the dialogue-stream may include timestamp of receipt of each voicecommand and text commands for each voice command. The dialogue-streammay gather the text commands and time of receipt of each voice commandassociated with the respective text command into a single collectionorganized by the time of receipt.

The method 1000 may further include acknowledging receipt of theworkflow record 1004. In accordance with one aspect of the presentdisclosure, the supervisor device 150 may transmit an indication to theuser device 110 or server 160 from which the workflow record wasreceived. In an embodiment, the indication may indicate that theworkflow record was received at the supervisor device 150.

The method 1000 may further include creating an alarm associated with atask performed in the workflow record 1006. In an embodiment, the alarmmay notify the supervisor device 150 that a user has exceeded a setduration of the task. In some embodiments, the set duration may bedetermined by the user device 110, supervisor device 150, and/or theserver 160 by calculating an average duration of the task as performedby users. In some embodiments, the user device 110, supervisor device150, and/or the server 160 may determine the average duration of tasksassociated with the voice commands by averaging the duration of eachtask over a stastically significant number of repetitions of the task.

Further, the method 1000 may include initiating an action to improve theworkflow record 1008. In accordance with one aspect of the presentdisclosure, the supervisor device 150 may further include adding a noteto the workflow record recording the initiated action. In someembodiments, the note can be time stamped and may be included in asummary of the worker's record. The note may allow a supervisor and/orthe worker to observe the effect of the action taken. For example, if aworker in an assembly area may be lagging in assembling the products, asreflected by the workflow record of the user, the supervisor and/or thesupervisor device 150 may initiate an appropriate action to improve theworkflow record. By way of an example, the initiated action may be toenroll the worker in the next available training slot for assembling thesaid product. The supervisor and/or the supervisor device 150 mayfurther add a note to the workflow record of the worker stating“Re-training Initiated.” In some embodiments, the user device 110,supervisor device 150, the server 160, and/or any other associateddevice may initiate an action to improve the workflow record bynotifying a user that the user is inefficient by transmitting anindication to the user device. In some embodiments, the action toimprove the workflow record may include requesting additional materials,workers, or a combination thereof to perform the task. In this regard,the user device 110, supervisor device 150, the server 160, and/or anyother associated device may transmit an indication to a serverindicating that additional materials, workers, etc. are needed toperform the task. Thus, based on the method 1000, as described above,inefficient workers may be readily identified and appropriate action maybe taken to ensure that worker productivity and efficiency is improved.

FIGS. 8-10 illustrate example flowcharts of the operations performed byan apparatus, such as user device of FIG. 1 and/or server of FIG. 1 , inaccordance with example embodiments of the present invention. It will beunderstood that each block of the flowcharts, and combinations of blocksin the flowcharts, may be implemented by various means, such ashardware, firmware, one or more processors, electronic module 502 and/orother devices associated with execution of software including one ormore computer program instructions. For example, one or more of theprocedures described above may be embodied by computer programinstructions. In this regard, the computer program instructions whichembody the procedures described above may be stored by a memory of anapparatus employing an embodiment of the present invention and executedby a processor in the apparatus. As will be appreciated, any suchcomputer program instructions may be loaded onto a computer or otherprogrammable apparatus (e.g., hardware) to produce a machine, such thatthe resulting computer or other programmable apparatus provides forimplementation of the functions specified in the flowcharts' block(s).These computer program instructions may also be stored in anon-transitory computer-readable storage memory that may direct acomputer or other programmable apparatus to function in a particularmanner, such that the instructions stored in the computer-readablestorage memory produce an article of manufacture, the execution of whichimplements the function specified in the flowcharts' block(s). Thecomputer program instructions may also be loaded onto a computer orother programmable apparatus to cause a series of operations to beperformed on the computer or other programmable apparatus to produce acomputer-implemented process such that the instructions which execute onthe computer or other programmable apparatus provide operations forimplementing the functions specified in the flowcharts' block(s). Assuch, the operations of FIGS. 8-10 when executed, convert a computer orprocessing electronic module 502 into a particular machine configured toperform an example embodiment of the present invention. Accordingly, theoperations of FIGS. 8-10 define an algorithm for configuring a computeror processor, to perform an example embodiment. In some cases, a generalpurpose computer may be provided with an instance of the processor whichperforms the algorithm of FIGS. 8-10 to transform the general purposecomputer into a particular machine configured to perform an exampleembodiment.

Accordingly, blocks of the flowchart support combinations of means forperforming the specified functions and combinations of operations forperforming the specified functions. It will also be understood that oneor more blocks of the flowcharts', and combinations of blocks in theflowchart, can be implemented by special purpose hardware-based computersystems which perform the specified functions, or combinations ofspecial purpose hardware and computer instructions.

In some example embodiments, certain ones of the operations herein maybe modified or further amplified as described below. Moreover, in someembodiments additional optional operations may also be included. Itshould be appreciated that each of the modifications, optional additionsor amplifications described herein may be included with the operationsherein either alone or in combination with any others among the featuresdescribed herein.

The foregoing method descriptions and the process flow diagrams areprovided merely as illustrative examples and are not intended to requireor imply that the steps of the various embodiments must be performed inthe order presented. As will be appreciated by one of skill in the artthe order of steps in the foregoing embodiments may be performed in anyorder. Words such as “thereafter,” “then,” “next,” etc. are not intendedto limit the order of the steps; these words are simply used to guidethe reader through the description of the methods. Further, anyreference to claim elements in the singular, for example, using thearticles “a,” “an” or “the” is not to be construed as limiting theelement to the singular.

The various illustrative logical blocks, modules, circuits, andalgorithm steps described in connection with the embodiments disclosedherein may be implemented as electronic hardware, computer software, orcombinations of both. To clearly illustrate this interchangeability ofhardware and software, various illustrative components, blocks, modules,circuits, and steps have been described above generally in terms oftheir functionality. Whether such functionality is implemented ashardware or software depends upon the particular application and designconstraints imposed on the overall system. Skilled artisans mayimplement the described functionality in varying ways for eachparticular application, but such implementation decisions should not beinterpreted as causing a departure from the scope of the presentinvention.

The hardware used to implement the various illustrative logics, logicalblocks, modules, and circuits described in connection with the aspectsdisclosed herein may be implemented or performed with a general purposeprocessor, a digital signal processor (DSP), an application specificintegrated circuit (ASIC), a field programmable gate array (FPGA) orother programmable logic device, discrete gate or transistor logic,discrete hardware components, or any combination thereof designed toperform the functions described herein. A general-purpose processor maybe a microprocessor, but, in the alternative, the processor may be anyconventional processor, controller, microcontroller, or state machine. Aprocessor may also be implemented as a combination of computing devices,e.g., a combination of a DSP and a microprocessor, a plurality ofmicroprocessors, one or more microprocessors in conjunction with a DSPcore, or any other such configuration. Alternatively, some steps ormethods may be performed by electronic module 502 that is specific to agiven function.

In one or more exemplary aspects, the functions described may beimplemented in hardware, software, firmware, or any combination thereof.If implemented in software, the functions may be stored as one or moreinstructions or code on a non-transitory computer-readable medium ornon-transitory processor-readable medium. The steps of a method oralgorithm disclosed herein may be embodied in a processor-executablesoftware module (or processor-executable instructions) which may resideon a non-transitory computer-readable or processor-readable storagemedium. Non-transitory computer-readable or processor-readable storagemedia may be any storage media that may be accessed by a computer or aprocessor. By way of example but not limitation, such non-transitorycomputer-readable or processor-readable media may include RAM, ROM,EEPROM, FLASH memory, CD-ROM or other optical disk storage, magneticdisk storage or other magnetic storage devices, or any other medium thatmay be used to store desired program code in the form of instructions ordata structures and that may be accessed by a computer. Disk and disc,as used herein, includes compact disc (CD), laser disc, optical disc,digital versatile disc (DVD), floppy disk, and blu-ray disc where disksusually reproduce data magnetically, while discs reproduce dataoptically with lasers. Combinations of the above are also includedwithin the scope of non-transitory computer-readable andprocessor-readable media. Additionally, the operations of a method oralgorithm may reside as one or any combination or set of codes and/orinstructions on a non-transitory processor-readable medium and/orcomputer-readable medium, which may be incorporated into a computerprogram product.

While various embodiments in accordance with the principles disclosedherein have been shown and described above, modifications thereof may bemade by one skilled in the art without departing from the spirit and theteachings of the disclosure. The embodiments described herein arerepresentative only and are not intended to be limiting. Manyvariations, combinations, and modifications are possible and are withinthe scope of the disclosure. Alternative embodiments that result fromcombining, integrating, and/or omitting features of the embodiment(s)are also within the scope of the disclosure. Accordingly, the scope ofprotection is not limited by the description set out above, but isdefined by the claims which follow, that scope including all equivalentsof the subject matter of the claims. Each and every claim isincorporated as further disclosure into the specification and the claimsare embodiment(s) of the present invention(s). Furthermore, anyadvantages and features described above may relate to specificembodiments, but shall not limit the application of such issued claimsto processes and structures accomplishing any or all of the aboveadvantages or having any or all of the above features.

In addition, the section headings used herein are provided forconsistency with the suggestions under 37 C.F.R. 1.77 or to otherwiseprovide organizational cues. These headings shall not limit orcharacterize the invention(s) set out in any claims that may issue fromthis disclosure. For instance, a description of a technology in the“Background” is not to be construed as an admission that certaintechnology is prior art to any invention(s) in this disclosure. Neitheris the “Summary” to be considered as a limiting characterization of theinvention(s) set forth in issued claims. Furthermore, any reference inthis disclosure to “invention” in the singular should not be used toargue that there is only a single point of novelty in this disclosure.Multiple inventions may be set forth according to the limitations of themultiple claims issuing from this disclosure, and such claimsaccordingly define the invention(s), and their equivalents, that areprotected thereby. In all instances, the scope of the claims shall beconsidered on their own merits in light of this disclosure, but shouldnot be constrained by the headings set forth herein.

Also, techniques, systems, subsystems, and methods described andillustrated in the various embodiments as discrete or separate may becombined or integrated with other systems, modules, techniques, ormethods without departing from the scope of the present disclosure.Other items shown or discussed as directly coupled or communicating witheach other may be indirectly coupled or communicating through someinterface, device, or intermediate component, whether electrically,mechanically, or otherwise. Other examples of changes, substitutions,and alterations are ascertainable by one skilled in the art and could bemade without departing from the spirit and scope disclosed herein.

Many modifications and other embodiments of the inventions set forthherein will come to mind to one skilled in the art to which theseinventions pertain having the benefit of teachings presented in theforegoing descriptions and the associated drawings. Although the figuresonly show certain components of the apparatus and systems describedherein, it is understood that various other components may be used inconjunction with the supply management system. Therefore, it is to beunderstood that the inventions are not to be limited to the specificembodiments disclosed and that modifications and other embodiments areintended to be included within the scope of the appended claims. Forexample, the various elements or components may be combined orintegrated in another system or certain features may be omitted or notimplemented. Moreover, the steps in the method described above may notnecessarily occur in the order depicted in the accompanying diagrams,and in some cases one or more of the steps depicted may occursubstantially simultaneously, or additional steps may be involved.Although specific terms are employed herein, they are used in a genericand descriptive sense only and not for purposes of limitation.

We claim:
 1. A voice controlled apparatus, comprising: a processorconfigured to: transform a plurality of voice commands into a pluralityof text commands; create a first dialogue-stream based on the pluralityof text commands, wherein the first dialogue-stream includes one or moretags that are associated with the plurality of text commands of thefirst dialogue-stream, and wherein each of the one or more tags of thefirst dialogue-stream indicates that an associated text command of thefirst dialogue-stream corresponds to a user-initiated command; recognizethe one or more tags in the first dialogue-stream; determine items inthe first dialogue-stream that are not associated with the one or morerecognized tags; remove the items in the first dialogue-stream; andcreate a second dialogue-stream based on the removal of the items. 2.The apparatus of claim 1, wherein the first dialogue-stream includes theplurality of text commands and a time of receipt of each voice commandof the plurality of voice commands associated with the respective textcommand and is gathered into a single collection organized by the timeof receipt.
 3. The apparatus of claim 1, wherein the processor isfurther configured to receive a plurality of manually-inputted buttoncommands, incorporate the plurality of button commands into theplurality of text commands by identifying one or more points in datastored in a database related to each of the plurality of buttoncommands, and incorporate the plurality of text commands from theplurality of button commands into the second dialogue-stream.
 4. Theapparatus of claim 1, wherein the processor is further configured toprovide speech output by converting one or more points in data to one ormore audio signals to be converted to the speech output via a speaker.5. The apparatus of claim 1, wherein the apparatus further comprises amicrophone and a speaker.
 6. The apparatus of claim 1, wherein thesecond dialogue-stream is transmitted to a supervisor device.
 7. Theapparatus of claim 1, wherein the second dialogue-stream is further usedto prepare one or more alarms associated with a maximum duration of aparticular task of a workflow record that is created from the seconddialogue-stream.
 8. The apparatus of claim 1, wherein the seconddialogue-stream is further used to determine a rate of interaction of auser with the apparatus.
 9. The apparatus of claim 1, wherein the seconddialogue-stream is further used to identify periods of non-activity of auser with the apparatus.
 10. A method comprising: transforming aplurality of voice commands into a plurality of text commands; creatinga first dialogue-stream based on the plurality of text commands, whereinthe first dialogue-stream includes one or more tags that are associatedwith the plurality of text commands of the first dialogue-stream, andwherein each of the one or more tags of the first dialogue-streamindicates that an associated text command of the first dialogue-streamcorresponds to a user-initiated command; recognizing the one or moretags in the first dialogue-stream; determining items in the firstdialogue-stream that are not associated with the one or more recognizedtags; removing the items in the first dialogue-stream; and creating asecond dialogue-stream based on the removal of the items.
 11. The methodof claim 10, further comprising gathering the plurality of text commandsand a time of receipt of each voice command of the plurality of voicecommands associated with the respective text command into a singlecollection organized by the time of receipt.
 12. The method of claim 10,wherein the method further comprises: determining a rate of interactionof a user by calculating a number of text commands over a period of timein the second dialogue-stream.
 13. The method of claim 10, furthercomprising creating a workflow record for a user associated with thefirst dialogue-stream and the second dialogue-stream by: identifying atleast one task associated with any one of the plurality of voicecommands from the user based on the text command associated with therespective voice command; and determining a duration of the at least onetask based on a time associated with each text command, wherein theworkflow record is used to generate an efficiency metric.
 14. The methodof claim 13, wherein identifying the at least one task associated withany one of the plurality of voice commands from the user based on thetext command associated with the respective voice command comprises:determining a current state of a user device associated with the user;determining a next state of the user device associated with the user;identifying a trigger to move to the next state of the user device; andadding the at least one task to the workflow record after identifyingthe trigger, wherein the current state is determined by the earliestreceived voice command and associated text command or a voice commandreceived from the user device that has not been associated with at leastone task, the next state is determined by identifying a task intended tofollow the current state from a database, and the trigger comprises adialogue of interest related to the next state.
 15. The method of claim13, wherein determining the duration of the at least one task comprisesidentifying a text command or a voice command associated with a start ofthe at least one task in a database, identifying a text command or avoice command associated with an end of the at least one task in thedatabase and determining a time between the start of the at least onetask and the end of the at least one task.
 16. The method of claim 13,wherein the method further comprises: determining an average duration ofa plurality of tasks associated with the plurality of voice commands byaveraging the duration of each task over a stastically significantnumber of repetitions of the task.
 17. The method of claim 13, whereinthe method further comprises: creating at least one alarm associatedwith the at least one task, wherein the alarm is configured to notify asupervisor device that the user has exceeded a maximum duration of theat least one task by causing an indication to be transmitted to thesupervisor device when the duration of the at least one task exceeds themaximum duration of the at least one task in the workflow record. 18.The method of claim 13, wherein the method further comprises: causingthe workflow record for the particular user to be transmitted to asupervisor device, wherein the supervisor device is controlled by asupervisor associated with the user.
 19. The method of claim 13, whereinthe method further comprises: receiving a plurality of dialogue-streamsfrom a plurality of user devices, wherein each dialogue-stream comprisesa plurality of text commands and a time associated with each textcommand, the plurality of text commands formed by transforming aplurality of voice commands into the plurality of text commands byconverting audio signals associated with the plurality of voice commandsto one or more points in data that are stored in the database or anotherdatabase; creating a plurality of workflow records for the plurality ofusers by identifying at least one task for at least one voice command ineach of the plurality of workflow records and determining a duration ofeach task of each workflow record of the plurality of workflow records;and comparing the duration of each task of each workflow record of theplurality of workflow records.
 20. The method of claim 19, wherein themethod further comprises: filtering the plurality of workflow recordsfor a single task; and determining at least one of an average durationof the single task over the plurality of workflow records, a minimumduration of the single task over the plurality of workflow records, or amaximum duration of the single task over the plurality of workflowrecords.
 21. The method of claim 20, wherein the method furthercomprises: creating at least one alarm associated with the single task,wherein the alarm is configured to notify a supervisor device that atleast one user has exceeded a set duration of the single task by causingan indication to be transmitted to the supervisor device when theduration of the single task exceeds the set duration in the workflowrecord of the at least one user.
 22. The method of claim 21, wherein themethod further comprises causing the at least one alarm to betransmitted to the user device.
 23. The method of claim 21, wherein theset duration is determined by calculating an average duration of thesingle task as performed over a stastically significant number ofrepetitions of the single task.
 24. The method of claim 13, wherein themethod further comprises identifying a period of non-activity of theuser by identifying a period of time where a rate of interactiondecreases to a defined value and causing an indication to be transmittedto a supervisor device notifying of the occurrence of the period ofnon-activity.
 25. The method of claim 13, further comprising initiatingan action to improve the workflow record and incorporating at least onenote to the workflow record recording the initiated action.
 26. Themethod of claim 25, wherein the action to improve the workflow recordcomprises causing an indication to be transmitted to a user deviceoperated by the user, the indication indicating that the user isinactive.
 27. The method of claim 25, wherein the action to improve theworkflow record comprises requesting additional materials, workers, or acombination thereof to perform a task by causing an indication to betransmitted to a server, the indication indicating that additionalmaterials, workers, or a combination thereof are needed to perform thetask.
 28. The method of claim 13, wherein the workflow record comprisesa plurality of tasks associated with the plurality of text commands andeach of the plurality of tasks is associated with a duration of therespective task.